Judges describe a campaign of firings and interference which threatens the system's independence. On a Thursday morning last month, Patrick O'Brien, a federal immigration judge, walked into his courtroom in downtown San Francisco. He was scheduled for a master-calendar hearing, a roll call, essentially, to get cases ready for trial. O'Brien was wearing a matte-black robe that seemed to absorb the artificial light overhead. He took his seat, scanned the room, and angled himself toward a computer monitor. The court was leanly staffed. There was a judicial clerk but no bailiff or stenographer. Opposite the judge were tables for the prosecution--the Department of Homeland Security--and for the respondent, a succession of immigrants who were applying for asylum. A Spanish interpreter appeared as a faceless box on a big screen. About ten people, all Latino, sat in wooden pews, gripping folders full of esoteric documents.

In this study, we introduce a novel two-pronged scoring system to measure individual and systemic bias in immigration courts under the U.S. Executive Office of Immigration Review (EOIR). We analyze nearly 6 million immigration court proceedings and 228 case features to build on prior research showing that U.S. asylum decisions vary dramatically based on factors that are extraneous to the merits of a case. We close a critical gap in the literature of variability metrics that can span space and time. Using predictive modeling, we explain 58.54% of the total decision variability using two metrics: partisanship and inter-judge cohort consistency. Thus, whether the EOIR grants asylum to an applicant or not depends in majority on the combined effects of the political climate and the individual variability of the presiding judge - not the individual merits of the case. Using time series analysis, we also demonstrate that partisanship increased in the early 1990s but plateaued following the turn of the century. These conclusions are striking to the extent that they diverge from the U.S. immigration system's commitments to independence and due process. Our contributions expose systemic inequities in the U.S. asylum decision-making process, and we recommend improved and standardized variability metrics to better diagnose and monitor these issues.